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PARTICLE EXTRACTION FOR AUTOMATIC FLOW MICROSCOPE 

This application claims the benefit of U.S. Provisional Application No. 60/427,466, 
5 filed November 18,2002. 



FIELD OF THE INVENTION 

The present invention relates to methods and systems for analyzing particles in a 
10 dilute fluid sample, and more particularly to a method and apparatus for automatically 
locating the boundary of an object in a field of view. 



BACKGROUND OF THE INVENTION 

Methods and systems for analyzing particles in a dilute fluid sample are well known, 
15 as disclosed in U.S. Patents 4,338,024 and 4,393,466. Existing flow microscope particle 
analyzers use an image acquisition device and software that detects particles based on their 
brightness difference with the background. U.S. Patents 4,538,299 and 5,625,709 are 
examples of such analyzers. Unfortunately, currently employed systems and methods often 
cannot efficiently detect low contrast particles, and often identify different parts of the same 
20 object as different objects, resulting in incorrect classification and reported element quantity. 



SUMMARY OF THE INVENTION 

The present invention is a method of accurately identifying particle images, and 
producing image segments each containing the image of a single particle. 

25 The present invention is a method for automatically locating a boundary of an object 

of interest in a field of view, which includes forming an electronic image of the field of view 
containing the object, wherein the electronic image is formed of a plurality of image pixels, 
identifying groups of the image pixels that represent edge segments of the object, forming 
patches around the image pixel groups, wherein each patch is dimensioned and positioned to 

30 entirely contain one of the image pixel groups, and performing a patch merge process that 
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merges any two of the patches together that meet a predetermined proximity threshold 
relative to each other to form a merged patch that is dimensioned and positioned to entirely 
contain the two merged patches. The merge process continues for any of the patches and the 
merged patches meeting the predetermined proximity threshold until none of the patches and 
5 the merged patches meet the predetermined proximity threshold. 

The present invention is also an apparatus for automatically locating a boundary of an 
object of interest in a field of view, which includes an imaging system for forming an 
electrical image of the field of view containing the object, wherein the electronic image is 
formed of a plurality of image pixels, and at least one processor. The at least one processor 

10 identifies groups of the image pixels that represent edge segments of the object, forms 

patches around the image pixel groups, wherein each patch is dimensioned and positioned to 
entirely contain one of the image pixel groups, and performs a patch merge process that 
merges any two of the patches together that meet a predetermined proximity threshold 
relative to each other to form a merged patch that is dimensioned and positioned to entirely 

15 contain the two merged patches. The merge process continues for any of the patches and the 
merged patches meeting the predetermined proximity threshold until none of the patches and 
the merged patches meet the predetermined proximity threshold. 

Other objects and features of the present invention will become apparent by a review 
of the specification, claims and appended figures. 

20 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic diagram of a particle analyzer employing the method of the 
present invention. 

Fig. 2 is a flow chart showing the method steps of the present invention. 
25 Fig. 3 is a flow chart showing the method steps for setting the background level. 

Fig. 4 is a flow chart showing the method steps for creating binary images. 

Fig. 5 is an example of a pixel value array for an exemplary binary image derived 
from method steps of Fig. 4. 

Fig. 6 is an example of the pixel value array of Fig. 5 after edge detection. 
30 Fig. 7 is an example of the pixel value array of Fig. 6 after the formation of patches. 
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Figs. 8A-8C are examples of the pixel value array of Fig. 7 illustrating the merging of 
patches around a single particle. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

5 The method of the present invention enhances the detection of particles to allow low 

contrast particle detection and object parts combination. The method includes 5 basic steps, 
and can be employed using a conventional particle analyzer having an imaging system 2 and 
a processor 4, as schematically illustrated in Fig. 1. 

10 Imaging system and processor 

Imaging system 2 is used to produce images of fields of view of a sample containing 
the particles of interest. Imaging system 2 is preferably a well known flow microscope as 
described in U.S. Patents 4,338,024, 4,393,466, 4,538,299 and 4,612,614, which are all 
hereby incorporated herein by reference. Such systems include a flow cell 10, a microscope 

15 12, and a camera 14, as shown in Fig. 1. Specimen fluid containing the particles of interest is 
passed through an examination area of the flow cell 10, whereby images of the particles are 
viewable through the flow microscope 12. The camera 14 (which is preferably a CCD 
camera) captures images of successive fields of view of the particles via the microscope 12, 
as the particles flow through the flow cell 10, and converts them to digital particle images. 

20 Each of the digital particle images taken by the camera 14 comprise thousands or even 

millions of individual pixels. A light source 16 (e.g. strobe) is preferably used to illuminate 
(by front and/or back lighting) the examination area of the flow cell 10. It should be noted 
that the present invention can also be applied to an imaging system that analyzes non-flowing 
specimen fluid (e.g. specimen fluid placed on an examination slide). 

25 Processor 4 can be any microprocessor and/or computer system, or a plurality of 

microprocessors and/or computer systems, capable of processing the digital particle images 
as described below. Examples of such processors include, but are not limited to, data 
processors, DSP's (digital signal processors), microcontrollers, and computer system 
processors, each of which can be CISC and/or RISC type. 

30 
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Method of particle detection enhancement 

There are five basic steps of the particle detection method of the present invention, as 
illustrated in Fig. 2: 1) Setting Background Level, 2) Creating Binary Image, 3) Detecting 
Particle Edges, 4) Forming Patches, and 5) Merging Patches. 

5 

Step 1 : Setting Background Level 
Fig. 3 illustrates the four steps of setting a background level. This process not only 
creates a median background level for each pixel location, but does so in a manner that 
compensates for any fluctuations in the background illumination from image to image (e.g. 
10 different intensities from strobe 16 which can adversely affect the resulting accuracy of the 
system). 

First (in step 1-1), a transparent fluid is sent through the flow cell 10 (or placed on 
examination slides), where N images of the transparent fluid are taken by camera 14 and 
digitized to create N digitized background images. As a non-limiting example, N can equal 

15 3. Second (in step 1-2), histograms of the digital background image pixel values are made 
for each of the N images, where the peak value of each histogram corresponds to an average 
pixel value for that image. Third (in step 1-3), a predetermined "standardization" pixel value 
is selected, and each image is standardized to that pixel value by adding to or subtracting 
from all its pixel values, so that all of the N images have a histogram peak at the 

20 predetermined standardization pixel value. For example, if the predetermined 

standardization pixel value is 215, and one of the N images has a histogram peak at 210, then 
' all pixels in that image are increased by a value of 5. If another of the N images has a 
histogram peak at 190, then all pixel values in that image are increased by 25. Thus, all N 
images are standardized to a single pixel value, which compensates for fluctuations in 

25 background illumination from image to image. Fourth (in step 1-4), for each pixel location 
of the digital images, a median pixel value is determined from the pixel values in all of the N 
images (as corrected by the standardization step 1-3), where a median background level 
image is created on a pixel location by pixel location basis. Thus, each pixel value of the 
median background level image represents the corrected median value for that pixel location 

30 from all of the standardized N images, where N can be any number of images, including 1 . 
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Preferably, the predetermined standardization pixel value is selected to be as high as 
possible without causing pixel saturation. Thus, in a system where the pixel values can range 
from 0 to 255, the predetermined standardization pixel value is selected to prevent any of 
pixel values in the background images from reaching 255. 
5 By compensating for background illumination (e.g. flash to flash) variations, it allows 

the system to operate with a lower threshold for distinguishing between background pixels 
and pixels that accurately reflect the particle edges, as discussed further below. It should be 
noted that for systems having minimal background illumination fluctuations from image to 
image, the illumination compensation of steps 1-2 and 1-3 can be omitted, whereby the 
10 median background level image is calculated in step 1-4 directly from measured pixel values 
of the N images taken in step 1-1 (without any standardization thereof), even where N equal 
1. If N equals 1, then the median pixel values are the measured pixel values. 

Step 2: Binary Image Creation 

15 Fig. 4 illustrates the steps for creating binary images. First (step 2-1), specimen fluid 

containing the particles of interest is sent through the flow cell (or positioned on examination 
slides), where images of the specimen fluid (and the particles therein) are taken by the 
camera 14 and digitized to form digital specimen images. Preferably but not necessarily, the 
illumination compensation discussed above (steps 1-2 and 1-3) is performed on each of the 

20 specimen images (to compensate for illumination differences from specimen image to 
specimen image). Specifically, histograms of the pixel values are made for each of the 
specimen images, where the peak value of each histogram corresponds to an average pixel 
value for that specimen image (step 2-2). A predetermined standardization pixel value is 
selected (preferably using the highest value possible while avoiding pixel saturation), and 

25 each specimen image is standardized to that pixel value by adding to or subtracting from all 
its pixel values as described above, so that all of the specimen images have a histogram peak 
at the predetermined standardization pixel value (step 2-3). Preferably, the predetermined 
standardization pixel value for the specimen image is the same as that used for standardizing 
the background images. Finally, a binary image is created for each specimen image on a 

30 pixel by pixel basis (step 2-4), which highlights only those pixels for which the difference 
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between the median background level image value and the specimen image value exceeds a 
predetermined threshold value X. 

Fig. 5 illustrates an example of a binary image, where the binary image pixel values 
are assigned one value (e.g. a "1") where the difference between corresponding specimen 
5 image pixel values and median background level image pixel values exceed the threshold 
value X, with the remaining binary image pixels assigned a second pixel value (e.g. a "0"). It 
should be noted that for specimen images that are backlit, pixels corresponding to the 
background are lighter (have a greater intensity value) than those pixels corresponding to 
portions of the particles. 

10 The predetermined threshold value X is selected to be as small as possible, without 

incurring too much noise, to maximize the sensitivity of the system for low contrast particle 
images. If the threshold is set too high, the system will erroneously identify, as part of the 
background, some pixels that in fact correspond to portions of the particles. If the threshold 
is set too low, the system will erroneously identify, as part of the particles, some pixels that 

15 in fact correspond to portions of the background. For best results, the ideal threshold value X 
should be empirically determined for each system. 



20 Step 3: Edge Detection. 

For each of the specimen images, edges of the various particles therein are 
determined using the corresponding binary image created in Step 3. Particle edge detection 
in imaging systems is well known, and involves the process of identifying those pixels of the 
specimen images that correspond to the edges of the particles in those images. For example, 

25 pixels corresponding to the particle edges are often identified as those having neighboring 
pixels on one side that correspond to the image's background, and having neighboring pixels 
on the other side that correspond to the particle interior. Typical edge detection techniques 
start by identifying one or more edge pixels, and then search for more such edge pixels 
adjacent those already so identified. Exemplary particle edge tracing techniques are 

30 disclosed in U.S. Patent Numbers 5,626,709 and 4,538,299, which are incorporated herein by 
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reference. Once the edge pixels are identified, those pixels are distinguished from the rest of 
the image (which corresponds to background and particle interior portions of the image). For 
example, the binary image of Fig. 5 can be processed using an edge detection technique such 
that those pixels that are identified as edge pixels are left assigned as 'Ts", with the 
5 remaining pixels all assigned as "O's", as illustrated in Fig. 6. The result is a plurality of 
pixels that define particle edges or edge segments (an edge segment is formed by a group of 
edge pixels that continuously define at least a portion of a particle edge). 

Step 4: Patch Formation 

10 For each specimen image, rectangular patches are created around each detected 

particle edge or edge segment, where each patch is dimensioned as small as possible while 
still containing the entire detected particle edge or edge segment. A simple exemplary 
creation of the rectangular patches for the binary image of Fig. 6 is illustrated in Fig. 7, 
where four distinct edge segments for one particle result in four rectangular patches (PI, P2, 

15 P3andP4). 

As an optional step, each patch can be enlarged by expanding each of its walls away 
from the patch center by the same predetermined amount. For example, if a patch size is 10 
X 20 pixels, and the predetermined amount is 5 pixels, then each patch wall is moved back 
by 5 pixels, leaving a patch size of 20 X 30. The optimal predetermined amount of 

20 enlargement, which produces the most accurate particle identification results, will vary 

depending on system design, and can be empirically determined. In a particle analyzer that 
has been reduced to practice, the optimal predetermined amount of enlargement was 
determined by trial and error to be 12 pixels (out of approximately 1 .4 million total pixels in 
the entire image), such that a patch size of 10 X 20 is expanded to a patch size of 34 X 44. 

25 The patches are preferably enlarged before proceeding to the fifth and final step. 



Step 5: Patch Merge 

The final step is to successively merge overlapping patches to identify those patches 
(and the edge segments therein) that belong to the same particle. The merge process begins 
30 by merging together any of the patches created (and possibly expanded) from Step 4 that 
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overlap each other by sharing more than a predetermined number of pixels Y. Thus, for any 
two patches that meet that criteria, the two patches are merged into a single patch, where the 
single patch represents the smallest rectangle that can entirely contain both of the 
overlapping patches. This merging process continues using the original patches from Step 4, 
the merged patches of the present step, and/or any combinations of the two, until no further 
patches can be merged using the above described criteria. 

Figs. 8 A to 8C illustrate a simplistic example of the merging of patches from Fig. 7. 
In Fig. 7, two of the patches (PI and P2) are overlapping (by having two pixels in common). 
Assuming the predetermined number of pixels Y does not exceed two, patches PI and P2 are 
merged together as shown in Fig. 8 A into a single merged patch P5. Patch P5 now overlaps 
with patch P3, where Fig. 8B illustrates the merger of patches P5 and P3 to result in merged 
patch P6. In this particular example, no more merging takes place, because patch P6 does 
not overlap patch P4. Fig. 8B is a good example of why the optional patch enlargement step 
should be utilized, as it ensures patches adjacent but distinct from the other patches from the 
same particle are included in the final merged patch. Had patch enlargement of just a single 
pixel been performed before or even during the patch merge process, all four of the original 
patches (PI, P2, P3, P4) that represent four distinct particle edge segments of the same 
particle would have been properly included in the final merged patch P7, as illustrated in Fig. 
8C. 

Once patch merging is complete, those edge segments found within one of the final 
patches are associated with a single particle (as representing the boundary of that single 
particle), and any edge segment(s) found outside of that final patch are associated as either 
non-particles or part of an adjacent but distinct particle. At this time, any gaps between the 
edge segments within the one final. patch can be filled in to form a single and continuous 
particle edge. 

The optimal value of the predetermined number of pixels Y required to merge two 
patches will vary from system to system, and can be empirically determined. In a particle 
analyzer that has been reduced to practice, 50 pixels (out of 1.4 million total pixels in the 
entire image) was found to be an optimal predetermined number Y of shared pixels for 
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triggering a merging of overlapping patches. Once the patch merge process is completed, 
then each remaining patch should contain no more than the image of a single particle. 

By isolating each particle in distinct and known patches, the system can reliably avoid 
identifying different parts of the same particle as different particles, whereby any gaps in the 
5 particle edges can be filled in within each patch without the risk of bridging an edge portion 
of one particle to an edge portion of another particle. 

Patching merging as described above and illustrated in the drawings are described 
with respect to overlapping boundaries of the patches. However, patch merging can be 
performed in any manner that measures a proximity of the patches, where patches are merged 

1 0 if they meet a particular proximity threshold. Proximity can be measured simply by the 
overlap of patch boundaries containing distinct particle edge segments, as illustrated and 
described above. But in addition, other measures of proximity could be used, such as a 
measured distance or distances between patch boundary portions (e.g. distance between 
closest patch boundary portions, distance between furthest patch boundary portions, etc.), or 

15 a gravitational-like attraction criteria based on both patch sizes and separation distance(s) 
(e.g. consider overall size of patches and distance between patch centers, which is analogous 
to a mass and gravitational force analysis), where "big" patches might merge, but big and 
small patches with similar separation may not. Thus, the proximity threshold could be a 
particular number of pixels shared by two overlapping patches, or could be distance(s) 

20 between boundary segments of two patches, or could be a value based upon the sizes of the 
patches divided by the distance between the patch boundaries or the patch centers. 

It is to be understood that the present invention is not limited to the 
embodiment(s) described above and illustrated herein, but encompasses any and all 
variations falling within the scope of the appended claims. For example, as is apparent from 

25 the claims and specification, not all method steps need be performed in the exact order 

illustrated or claimed, but rather in any order that allows the proper isolation of particles in 
separate and distinct patches. Further, the shape of the patches need not necessarily be 
rectangular, and can vary in shape and orientation to optimize their merging behavior. Some 
shapes such as rectangles would more aggressively "capture" adjacent patches, while other 

30 shapes such as circles and ovals would minimize capture of adjacent objects, assuming that 
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all objects were inherently convex. In addition, while the system preferably automatically 
associates all the edge segments contained within the merged patch as representing the 
boundary of the object, it need not necessarily do so. Instead, the resulting edge segments 
contained with the merged patch can simply be presented to the user for analysis, or be used 
5 by the system for other analysis purposes. 
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